{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# License\n",
    "\n",
    "Copyright 2019 Hamaad Musharaf Shah\n",
    "\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n",
    "\n",
    "http://www.apache.org/licenses/LICENSE-2.0\n",
    "\n",
    "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Automatic feature engineering using Generative Adversarial Networks: Application to computer vision and synthetic financial transactions data\n",
    "## Author: Hamaad Shah\n",
    "\n",
    "---\n",
    "\n",
    "The purpose of deep learning is to learn a representation of high dimensional and noisy data using a sequence of differentiable functions, i.e., geometric transformations, that can perhaps be used for supervised learning tasks among others. It has had great success in discriminative models while generative models have fared worse due to the limitations of explicit maximum likelihood estimation (MLE). Adversarial learning as presented in the Generative Adversarial Network (GAN) aims to overcome these problems by using implicit MLE. \n",
    "\n",
    "We will use the MNIST computer vision dataset and a synthetic financial transactions dataset for an insurance task for these experiments. GAN is a remarkably different method of learning compared to explicit MLE. Our purpose will be to show that the representation learnt by a GAN can be used for supervised learning tasks such as image recognition and insurance loss risk prediction. In this manner we avoid the manual process of handcrafted feature engineering by learning a set of features automatically, i.e., representation learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Author: Hamaad Musharaf Shah.\n",
    "from PIL import Image\n",
    "\n",
    "from six.moves import range\n",
    "\n",
    "import os\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"-1\"\n",
    "import math\n",
    "import sys\n",
    "import importlib\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn import linear_model\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import MinMaxScaler, LabelBinarizer\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "import keras\n",
    "from keras import backend as bkend\n",
    "from keras.datasets import cifar10, mnist\n",
    "from keras import layers\n",
    "from keras.layers import Input, Dense, BatchNormalization, Dropout, Flatten, convolutional, pooling, Reshape\n",
    "from keras.layers.advanced_activations import LeakyReLU\n",
    "from keras.layers.convolutional import UpSampling2D, Conv2D\n",
    "from keras import metrics\n",
    "from keras.models import Model\n",
    "from keras.utils.generic_utils import Progbar\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.client import device_lib\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from plotnine import *\n",
    "import plotnine\n",
    "\n",
    "get_ipython().magic(\"matplotlib inline\")\n",
    "\n",
    "os.environ[\"KERAS_BACKEND\"] = \"tensorflow\"\n",
    "importlib.reload(bkend)\n",
    "\n",
    "print(device_lib.list_local_devices())\n",
    "\n",
    "mnist = mnist.load_data()\n",
    "(x_train, y_train), (x_test, y_test) = mnist\n",
    "x_train = np.reshape(x_train, [x_train.shape[0], x_train.shape[1] * x_train.shape[2]])\n",
    "x_test = np.reshape(x_test, [x_test.shape[0], x_test.shape[1] * x_test.shape[2]])\n",
    "y_train = y_train.ravel()\n",
    "y_test = y_test.ravel()\n",
    "x_train = x_train.astype(\"float32\")\n",
    "x_test = x_test.astype(\"float32\")\n",
    "x_train /= 255.0\n",
    "x_test /= 255.0\n",
    "\n",
    "scaler_classifier = MinMaxScaler(feature_range=(0.0, 1.0))\n",
    "logistic = linear_model.LogisticRegression(random_state=666, verbose=1)\n",
    "lb = LabelBinarizer()\n",
    "lb = lb.fit(y_train.reshape(y_train.shape[0], 1))\n",
    "\n",
    "dl4j_res_path = \"/Users/samson/Projects/gan_deeplearning4j/Java/src/main/resources/\"\n",
    "\n",
    "np.savetxt(fname=dl4j_res_path + \"mnist_train.csv\",\n",
    "           X=np.hstack([x_train, \n",
    "                        y_train.reshape([-1, 1])]),\n",
    "           fmt=\"%.2f\",\n",
    "           delimiter=\",\",\n",
    "           newline=\"\\n\",\n",
    "           header=\"\")\n",
    "\n",
    "np.savetxt(fname=dl4j_res_path + \"mnist_test.csv\",\n",
    "           X=np.hstack([x_test,\n",
    "                        y_test.reshape([-1, 1])]),\n",
    "           fmt=\"%.2f\",\n",
    "           delimiter=\",\",\n",
    "           newline=\"\\n\",\n",
    "           header=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generative Adversarial Network\n",
    "\n",
    "---\n",
    "\n",
    "There are 2 main components to a GAN, the generator and the discriminator, that play an adversarial game against each other. In doing so the generator learns how to create realistic synthetic samples from noise, i.e., the latent space $z$, while the discriminator learns how to distinguish between a real sample and a synthetic sample. \n",
    "\n",
    "The representation learnt by the discriminator can later on be used for other supervised learning tasks, i.e., automatic feature engineering or representation learning. This can also be viewed through the lens of transfer learning. A GAN can also be used for semi-supervised learning which we will get to in another paper where we will look into using variational autoencoders, ladder networks and adversarial autoencoders for this purpose.\n",
    "\n",
    "### Computer Vision\n",
    "\n",
    "---\n",
    "\n",
    "We will use the MNIST dataset for this purpose where the raw data is a 2 dimensional tensor of pixel intensities per image. The image is our unit of analysis: We will predict the probability of each class for each image. This is a multiclass classification task and we will use the accuracy score to assess model performance on the test fold.\n",
    "\n",
    "![](pixel_lattice.png)\n",
    "\n",
    "Some examples of handcrafted feature engineering for the computer vision task perhaps might be using Gabor filters.\n",
    "\n",
    "### Insurance\n",
    "\n",
    "---\n",
    "\n",
    "We will use a synthetic dataset where the raw data is a 2 dimensional tensor of historical policy level information per policy-period combination: Per unit this will be a 4 by 3 dimensional tensor, i.e., 4 historical time periods and 3 transactions types. The policy-period combination is our unit of analysis: We will predict the probability of loss for time period 5 in the future — think of this as a potential renewal of the policy for which we need to predict whether it would make a loss for us or not hence affecting whether we decided to renew the policy and / or adjust the renewal premium to take into account the additional risk. This is a binary class classification task and we will use the AUROC score to assess model performance.\n",
    "\n",
    "![](trans_lattice.png)\n",
    "\n",
    "Some examples of handcrafted feature engineering for the insurance task perhaps might be using column or row averages.\n",
    "\n",
    "The synthetic insurance financial transactions dataset was coded in R. All the rest of the work is done in Python.\n",
    "\n",
    "Please note the similarities between the raw data for the computer vision task and the raw data for the insurance task. Our main goal here is to learn a good representation of this raw data using automatic feature engineering via GANs.\n",
    "\n",
    "### Distributed Machine Learning via Deeplearning4j and Apache Spark\n",
    "\n",
    "---\n",
    "\n",
    "- We will use the Java library Deeplearning4j for the GANs along with Apache Spark for distributed machine learning via synchronous parameter averaging. \n",
    "- We use 2 external GPUs, i.e., GTX 1070, on a MacBook Pro. \n",
    "- We use IntelliJ for the Java IDE and Apache Maven for build automation.\n",
    "\n",
    "### Synchronous Parameter Averaging\n",
    "\n",
    "---\n",
    "\n",
    "Assume we have $n$ workers, $m$ observations per worker, $\\eta$ is the learning rate and the Mean Squared Error (MSE) as our objective function. Synchronous parameter averaging can be defined as follows.\n",
    "- **Map**: A local copy of the global model with weights $\\theta$ is sent to each worker and the local model is trained on the $m$ observations subset in each $n$ workers.\n",
    "- **Reduce**: The global model at the parameter server with weights $\\Theta$ is updated periodically by the average of the model weights taken from each of the $m$ workers.\n",
    "\n",
    "$$\n",
    "\\Theta \\leftarrow \\Theta + \\frac{\\eta}{n}\\sum_{i=1}^{n} \\left[\\nabla_{\\theta}\\text{MSE}\\right]_{i}\n",
    "$$\n",
    "\n",
    "### Generator\n",
    "\n",
    "---\n",
    "\n",
    "Assume that we have a prior belief on where the latent space $z$ lies: $p(z)$. Given a draw from this latent space the generator $G$, a deep learner parameterized by $\\theta_{G}$, outputs a synthetic sample.\n",
    "\n",
    "$$\n",
    "G(z|\\theta_{G}): z \\rightarrow x_{synthetic}\n",
    "$$ \n",
    "\n",
    "### Discriminator\n",
    "\n",
    "---\n",
    "\n",
    "The discriminator $D$ is another deep learner parameterized by $\\theta_{D}$ and it aims to classify if a sample is real or synthetic, i.e., if a sample is from the real data distribution,\n",
    "\n",
    "$$\n",
    "P_{\\text{data}}\n",
    "$$ \n",
    "\n",
    "or the synthetic data distribution.\n",
    "\n",
    "$$\n",
    "P_{G}\n",
    "$$\n",
    "\n",
    "Let us denote the discriminator $D$ as follows.\n",
    "\n",
    "$$\n",
    "D(x|\\theta_{D}): x \\rightarrow [0, 1]\n",
    "$$ \n",
    "\n",
    "Here we assume that the positive examples are from the real data distribution while the negative examples are from the synthetic data distribution.\n",
    "\n",
    "### Game: Optimality\n",
    "\n",
    "---\n",
    "\n",
    "A GAN simultaneously trains the discriminator to correctly classify real and synthetic examples while training the generator to create synthetic examples such that the discriminator incorrectly classifies real and synthetic examples. This 2 player minimax game has the following objective function.\n",
    "\n",
    "$$\n",
    "\\min_{G(z|\\theta_{G})} \\max_{D(x|\\theta_{D})} V(D(x|\\theta_{D}), G(z|\\theta_{G})) = \\mathbb{E}_{x \\sim p_{\\text{data}}(x)} \\log{D(x|\\theta_{D})} + \\mathbb{E}_{z \\sim p(z)} \\log{(1 - D(G(z|\\theta_{G})|\\theta_{D}))}\n",
    "$$\n",
    "\n",
    "Please note that the above expression is basically the objective function of the discriminator.\n",
    "\n",
    "$$\n",
    "\\mathbb{E}_{x \\sim p_{\\text{data}}(x)} \\log{D(x|\\theta_{D})} + \\mathbb{E}_{x \\sim p_{G}(x)} \\log{(1 - D(x|\\theta_{D}))}\n",
    "$$\n",
    "\n",
    "It is clear from how the game has been set up that we are trying to obtain a solution $\\theta_{D}$ for $D$ such that it maximizes $V(D, G)$ while simultaneously we are trying to obtain a solution $\\theta_{G}$ for $G$ such that it minimizes $V(D, G)$.\n",
    "\n",
    "We do not simultaneously train $D$ and $G$. We train them alternately: Train $D$ and then train $G$ while freezing $D$. We repeat this for a fixed number of steps.\n",
    "\n",
    "If the synthetic samples taken from the generator $G$ are realistic then implicitly we have learnt the distribution $P_{G}$. In other words, $P_{G}$ can be seen as a good estimation of $P_{\\text{data}}$. The optimal solution will be as follows.\n",
    "\n",
    "$$\n",
    "P_{G}=P_{\\text{data}}\n",
    "$$\n",
    "\n",
    "To show this let us find the optimal discriminator $D^\\ast$ given a generator $G$ and sample $x$. \n",
    "\n",
    "\\begin{align*}\n",
    "V(D, G) &= \\mathbb{E}_{x \\sim p_{\\text{data}}(x)} \\log{D(x|\\theta_{D})} + \\mathbb{E}_{x \\sim p_{G}(x)} \\log{(1 - D(x|\\theta_{D}))} \\\\\n",
    "&= \\int_{x} p_{\\text{data}}(x) \\log{D(x|\\theta_{D})} dx + \\int_{x} p_{G}(x) \\log{(1 - D(x|\\theta_{D}))} dx \\\\\n",
    "&= \\int_{x} \\underbrace{p_{\\text{data}}(x) \\log{D(x|\\theta_{D})} + p_{G}(x) \\log{(1 - D(x|\\theta_{D}))}}_{J(D(x|\\theta_{D}))} dx\n",
    "\\end{align*}\n",
    "\n",
    "Let us take a closer look at the discriminator's objective function for a sample $x$.\n",
    "\n",
    "\\begin{align*}\n",
    "J(D(x|\\theta_{D})) &= p_{\\text{data}}(x) \\log{D(x|\\theta_{D})} + p_{G}(x) \\log{(1 - D(x|\\theta_{D}))} \\\\\n",
    "\\frac{\\partial J(D(x|\\theta_{D}))}{\\partial D(x|\\theta_{D})} &= \\frac{p_{\\text{data}}(x)}{D(x|\\theta_{D})} - \\frac{p_{G}(x)}{(1 - D(x|\\theta_{D}))} \\\\\n",
    "0 &= \\frac{p_{\\text{data}}(x)}{D^\\ast(x|\\theta_{D^\\ast})} - \\frac{p_{G}(x)}{(1 - D^\\ast(x|\\theta_{D^\\ast}))} \\\\\n",
    "p_{\\text{data}}(x)(1 - D^\\ast(x|\\theta_{D^\\ast})) &= p_{G}(x)D^\\ast(x|\\theta_{D^\\ast}) \\\\\n",
    "p_{\\text{data}}(x) - p_{\\text{data}}(x)D^\\ast(x|\\theta_{D^\\ast})) &= p_{G}(x)D^\\ast(x|\\theta_{D^\\ast}) \\\\\n",
    "p_{G}(x)D^\\ast(x|\\theta_{D^\\ast}) + p_{\\text{data}}(x)D^\\ast(x|\\theta_{D^\\ast})) &= p_{\\text{data}}(x) \\\\\n",
    "D^\\ast(x|\\theta_{D^\\ast}) &= \\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)} \n",
    "\\end{align*}\n",
    "\n",
    "We have found the optimal discriminator given a generator. Let us focus now on the generator's objective function which is essentially to minimize the discriminator's objective function.\n",
    "\n",
    "\\begin{align*}\n",
    "J(G(x|\\theta_{G})) &= \\mathbb{E}_{x \\sim p_{\\text{data}}(x)} \\log{D^\\ast(x|\\theta_{D^\\ast})} + \\mathbb{E}_{x \\sim p_{G}(x)} \\log{(1 - D^\\ast(x|\\theta_{D^\\ast}))} \\\\\n",
    "&= \\mathbb{E}_{x \\sim p_{\\text{data}}(x)} \\log{\\bigg( \\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)}} \\bigg) + \\mathbb{E}_{x \\sim p_{G}(x)} \\log{\\bigg(1 - \\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)}\\bigg)} \\\\\n",
    "&= \\mathbb{E}_{x \\sim p_{\\text{data}}(x)} \\log{\\bigg( \\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)}} \\bigg) + \\mathbb{E}_{x \\sim p_{G}(x)} \\log{\\bigg(\\frac{p_{G}(x)}{p_{\\text{data}}(x) + p_{G}(x)}\\bigg)} \\\\\n",
    "&= \\int_{x} p_{\\text{data}}(x) \\log{\\bigg( \\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)}} \\bigg) dx + \\int_{x} p_{G}(x) \\log{\\bigg(\\frac{p_{G}(x)}{p_{\\text{data}}(x) + p_{G}(x)}\\bigg)} dx\n",
    "\\end{align*}\n",
    "\n",
    "We will note the Kullback–Leibler (KL) divergences in the above objective function for the generator.\n",
    "\n",
    "$$\n",
    "D_{KL}(P||Q) = \\int_{x} p(x) \\log\\bigg(\\frac{p(x)}{q(x)}\\bigg) dx\n",
    "$$\n",
    "\n",
    "Recall the definition of a $\\lambda$ divergence.\n",
    "\n",
    "$$\n",
    "D_{\\lambda}(P||Q) = \\lambda D_{KL}(P||\\lambda P + (1 - \\lambda) Q) + (1 - \\lambda) D_{KL}(Q||\\lambda P + (1 - \\lambda) Q)\n",
    "$$\n",
    "\n",
    "If $\\lambda$ takes the value of 0.5 this is then called the Jensen-Shannon (JS) divergence. This divergence is symmetric and non-negative.\n",
    "\n",
    "$$\n",
    "D_{JS}(P||Q) = 0.5 D_{KL}\\bigg(P\\bigg|\\bigg|\\frac{P + Q}{2}\\bigg) + 0.5 D_{KL}\\bigg(Q\\bigg|\\bigg|\\frac{P + Q}{2}\\bigg)\n",
    "$$\n",
    "\n",
    "Keeping this in mind let us take a look again at the objective function of the generator.\n",
    "\n",
    "\\begin{align*}\n",
    "J(G(x|\\theta_{G})) &= \\int_{x} p_{\\text{data}}(x) \\log{\\bigg( \\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)}} \\bigg) dx + \\int_{x} p_{G}(x) \\log{\\bigg(\\frac{p_{G}(x)}{p_{\\text{data}}(x) + p_{G}(x)}\\bigg)} dx \\\\\n",
    "&= \\int_{x} p_{\\text{data}}(x) \\log{\\bigg(\\frac{2}{2}\\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)}} \\bigg) dx + \\int_{x} p_{G}(x) \\log{\\bigg(\\frac{2}{2}\\frac{p_{G}(x)}{p_{\\text{data}}(x) + p_{G}(x)}\\bigg)} dx \\\\\n",
    "&= \\int_{x} p_{\\text{data}}(x) \\log{\\bigg(\\frac{1}{2}\\frac{1}{0.5}\\frac{p_{\\text{data}}(x)}{p_{\\text{data}}(x) + p_{G}(x)}} \\bigg) dx + \\int_{x} p_{G}(x) \\log{\\bigg(\\frac{1}{2}\\frac{1}{0.5}\\frac{p_{G}(x)}{p_{\\text{data}}(x) + p_{G}(x)}\\bigg)} dx \\\\\n",
    "&= \\int_{x} p_{\\text{data}}(x) \\bigg[ \\log(0.5) + \\log{\\bigg(\\frac{p_{\\text{data}}(x)}{0.5 (p_{\\text{data}}(x) + p_{G}(x))}} \\bigg) \\bigg] dx \\\\ &+ \\int_{x} p_{G}(x) \\bigg[\\log(0.5) + \\log{\\bigg(\\frac{p_{G}(x)}{0.5 (p_{\\text{data}}(x) + p_{G}(x))}\\bigg) \\bigg] } dx \\\\\n",
    "&= \\log\\bigg(\\frac{1}{4}\\bigg) + \\int_{x} p_{\\text{data}}(x) \\bigg[\\log{\\bigg(\\frac{p_{\\text{data}}(x)}{0.5 (p_{\\text{data}}(x) + p_{G}(x))}} \\bigg) \\bigg] dx \\\\ \n",
    "&+ \\int_{x} p_{G}(x) \\bigg[\\log{\\bigg(\\frac{p_{G}(x)}{0.5 (p_{\\text{data}}(x) + p_{G}(x))}\\bigg) \\bigg] } dx \\\\\n",
    "&= -\\log(4) + D_{KL}\\bigg(P_{\\text{data}}\\bigg|\\bigg|\\frac{P_{\\text{data}} + P_{G}}{2}\\bigg) + D_{KL}\\bigg(P_{G}\\bigg|\\bigg|\\frac{P_{\\text{data}} + P_{G}}{2}\\bigg) \\\\\n",
    "&= -\\log(4) + 2 \\bigg(0.5 D_{KL}\\bigg(P_{\\text{data}}\\bigg|\\bigg|\\frac{P_{\\text{data}} + P_{G}}{2}\\bigg) + 0.5 D_{KL}\\bigg(P_{G}\\bigg|\\bigg|\\frac{P_{\\text{data}} + P_{G}}{2}\\bigg)\\bigg) \\\\\n",
    "&= -\\log(4) + 2D_{JS}(P_{\\text{data}}||P_{G}) \n",
    "\\end{align*}\n",
    "\n",
    "It is clear from the objective function of the generator above that the global minimum value attained is $-\\log(4)$ which occurs when the following holds.\n",
    "\n",
    "$$\n",
    "P_{G}=P_{\\text{data}}\n",
    "$$\n",
    "\n",
    "When the above holds the Jensen-Shannon divergence, i.e., $D_{JS}(P_{\\text{data}}||P_{G})$, will be zero. Hence we have shown that the optimal solution is as follows.\n",
    "\n",
    "$$\n",
    "P_{G}=P_{\\text{data}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Game: Convergence\n",
    "\n",
    "---\n",
    "\n",
    "Assuming that the discriminator is allowed to reach its optimum given a generator, then $P_{G}$ can be shown to converge to $P_{\\text{data}}$. \n",
    "\n",
    "Consider the following objective function which has been previously shown to be convex with respect to $P_{G}$ as we found the global minimum at $-\\log(4)$.\n",
    "\n",
    "$$\n",
    "U(D^\\ast, P_{G}) = \\mathbb{E}_{x \\sim p_{\\text{data}}(x)} \\log{D^\\ast(x|\\theta_{D^\\ast})} + \\mathbb{E}_{x \\sim p_{G}(x)} \\log{(1 - D^\\ast(x|\\theta_{D^\\ast}))}\n",
    "$$\n",
    "\n",
    "Gradient descent is used by the generator to move towards the global minimum given an optimal discriminator. We will show that the gradient of the generator exists given an optimal discriminator, i.e., $\\nabla_{P_{G}} U(D^\\ast, P_{G})$, such that convergence of $P_{G}$ to $P_{\\text{data}}$ is guaranteed.\n",
    "\n",
    "Note that the following is a supremum of a set of convex functions where the set is indexed by the discriminator $D$: $U(D^\\ast, P_{G})=\\sup_{D} U(D, P_{G})$. Remember that the supremum is the least upper bound.\n",
    "\n",
    "Let us recall a few definitions regarding gradients and subgradients. A vector $g \\in \\mathbb{R}^K$ is a subgradient of a function $f: \\mathbb{R}^K \\rightarrow \\mathbb{R}$ at a point $x \\in \\mathbb{dom}(f)$ if $\\forall z \\in \\mathbb{dom}(f)$, the following relationship holds:\n",
    "\n",
    "$$\n",
    "f(z) \\geq f(x) + g^{T}(z - x)\n",
    "$$\n",
    "\n",
    "If $f$ is convex and differentiable then its gradient at a point $x$ is also the subgradient. Most importantly, a subgradient can exist even if $f$ is not differentiable.\n",
    "\n",
    "The subgradients of the supremum of a set of convex functions include the subgradient of the function at the point where the supremum is attained. As mentioned earlier, we have already shown that $U(D^\\ast, P_{G})$ is convex.\n",
    "\n",
    "\\begin{align*}\n",
    "&U(D^\\ast, P_{G})=\\sup_{D} U(D, P_{G}) \\\\\n",
    "&\\nabla_{P_{G}} \\sup_{D} U(D, P_{G}) \\in \\nabla_{P_{G}} U(D, P_{G}) \\\\\n",
    "&\\nabla_{P_{G}} U(D^\\ast, P_{G}) \\in \\nabla_{P_{G}} U(D, P_{G})\n",
    "\\end{align*}\n",
    "\n",
    "The gradient of the generator, $\\nabla_{P_{G}} U(D^\\ast, P_{G})$, is used to make incremental improvements to the objective function of the generator, $U(D^\\ast, P_{G})$, given an optimal discriminator, $D^\\ast$. Therefore convergence of $P_{G}$ to $P_{\\text{data}}$ is guaranteed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results\n",
    "\n",
    "---\n",
    "\n",
    "In these experiments we show the ability of the generator to create realistic synthetic examples for the MNIST dataset and the insurance dataset. We use a 2-dimensional latent manifold.\n",
    "\n",
    "Finally we show that using the representation learnt by the discriminator we can attain competitive results to using other representation learning methods for the MNIST dataset and the insurance dataset such as a wide variety of autoencoders.\n",
    "\n",
    "### Results: Generating new data\n",
    "\n",
    "---\n",
    "\n",
    "![](DCGAN_Generated_Images.png)\n",
    "\n",
    "![](DCGAN_Generated_Lattices.png)\n",
    "\n",
    "### Results: GAN for representation learning\n",
    "\n",
    "---\n",
    "\n",
    "* The accuracy score for the MNIST classification task with DCGAN: 97.070000%.\n",
    "* The AUROC score for the insurance classification task with DCGAN: 91.632719%.\n",
    "\n",
    "### The insurance data: A closer look\n",
    "\n",
    "With image data we can perhaps judge qualitatively whether the generated data makes sense. For financial transactions data this is not possible. However let's have a look at an example of a generated transactions lattice. Please note that all financial transactions data has been transformed to lie between 0 and 1. \n",
    "\n",
    "![](DCGAN_Generated_Lattice_Example.png)\n",
    "\n",
    "If we use the same matplotlib code as applied to the image data to plot the above generated transactions lattice we get the following image. We can see that where we have the maximum value possible for a transaction, i.e., 1, that is colored as black, while where we have the minimum value possible for a transaction, i.e., 0, that is colored as white. Transactions values in between have some gray color.  \n",
    "\n",
    "![](DCGAN_Generated_Lattice_Example_Plotted.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Java code for computer vision task\n",
    "\n",
    "```java\n",
    "package org.deeplearning4j;\n",
    "\n",
    "import java.io.*;\n",
    "import java.util.*;\n",
    "\n",
    "import org.apache.spark.SparkConf;\n",
    "import org.apache.spark.api.java.JavaRDD;\n",
    "import org.apache.spark.api.java.JavaSparkContext;\n",
    "\n",
    "import org.datavec.api.records.reader.RecordReader;\n",
    "import org.datavec.api.records.reader.impl.csv.CSVRecordReader;\n",
    "import org.datavec.api.split.FileSplit;\n",
    "import org.datavec.api.util.ClassPathResource;\n",
    "\n",
    "import org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator;\n",
    "import org.deeplearning4j.nn.api.*;\n",
    "import org.deeplearning4j.nn.conf.GradientNormalization;\n",
    "import org.deeplearning4j.nn.conf.WorkspaceMode;\n",
    "import org.deeplearning4j.nn.conf.inputs.InputType;\n",
    "import org.deeplearning4j.nn.conf.layers.*;\n",
    "import org.deeplearning4j.nn.conf.NeuralNetConfiguration;\n",
    "import org.deeplearning4j.nn.conf.preprocessor.FeedForwardToCnnPreProcessor;\n",
    "import org.deeplearning4j.nn.graph.ComputationGraph;\n",
    "import org.deeplearning4j.nn.transferlearning.*;\n",
    "import org.deeplearning4j.nn.weights.*;\n",
    "import org.deeplearning4j.spark.api.TrainingMaster;\n",
    "import org.deeplearning4j.spark.impl.graph.SparkComputationGraph;\n",
    "import org.deeplearning4j.spark.impl.paramavg.ParameterAveragingTrainingMaster;\n",
    "import org.deeplearning4j.util.ModelSerializer;\n",
    "\n",
    "import org.nd4j.jita.conf.CudaEnvironment;\n",
    "import org.nd4j.linalg.activations.*;\n",
    "import org.nd4j.linalg.api.buffer.DataBuffer;\n",
    "import org.nd4j.linalg.api.ndarray.INDArray;\n",
    "import org.nd4j.linalg.dataset.DataSet;\n",
    "import org.nd4j.linalg.dataset.api.iterator.DataSetIterator;\n",
    "import org.nd4j.linalg.factory.Nd4j;\n",
    "import org.nd4j.linalg.learning.config.*;\n",
    "import org.nd4j.linalg.lossfunctions.*;\n",
    "\n",
    "import org.slf4j.Logger;\n",
    "import org.slf4j.LoggerFactory;\n",
    "\n",
    "public class dl4jGANComputerVision {\n",
    "    private static final Logger log = LoggerFactory.getLogger(dl4jGANComputerVision.class);\n",
    "\n",
    "    private static final int batchSizePerWorker = 200;\n",
    "    private static final int batchSizePred = 500;\n",
    "    private static final int labelIndex = 784;\n",
    "    private static final int numClasses = 10; // Using Softmax.\n",
    "    private static final int numClassesDis = 1; // Using Sigmoid.\n",
    "    private static final int numFeatures = 784;\n",
    "    private static final int numIterations = 10000;\n",
    "    private static final int numGenSamples = 10; // This will be a grid so effectively we get {numGenSamples * numGenSamples} samples.\n",
    "    private static final int numLinesToSkip = 0;\n",
    "    private static final int numberOfTheBeast = 666;\n",
    "    private static final int printEvery = 10;\n",
    "    private static final int saveEvery = 100;\n",
    "    private static final int tensorDimOneSize = 28;\n",
    "    private static final int tensorDimTwoSize = 28;\n",
    "    private static final int tensorDimThreeSize = 1;\n",
    "    private static final int zSize = 2;\n",
    "\n",
    "    private static final double dis_learning_rate = 0.002;\n",
    "    private static final double frozen_learning_rate = 0.0;\n",
    "    private static final double gen_learning_rate = 0.004;\n",
    "\n",
    "    private static final String delimiter = \",\";\n",
    "    private static final String resPath = \"/Users/samson/Projects/gan_deeplearning4j/outputs/computer_vision/\";\n",
    "    private static final String newLine = \"\\n\";\n",
    "    private static final String dataSetName = \"mnist\";\n",
    "\n",
    "    private static final boolean useGpu = true;\n",
    "\n",
    "    public static void main(String[] args) throws Exception {\n",
    "        new dl4jGANComputerVision().GAN(args);\n",
    "    }\n",
    "\n",
    "    private void GAN(String[] args) throws Exception {\n",
    "        for (int i = 0; i < args.length; i++) {\n",
    "            System.out.println(args[i]);\n",
    "        }\n",
    "\n",
    "        if (useGpu) {\n",
    "            System.out.println(\"Setting up CUDA environment!\");\n",
    "            Nd4j.setDataType(DataBuffer.Type.FLOAT);\n",
    "\n",
    "            CudaEnvironment.getInstance().getConfiguration()\n",
    "                    .allowMultiGPU(true)\n",
    "                    .setMaximumDeviceCache(2L * 1024L * 1024L * 1024L)\n",
    "                    .allowCrossDeviceAccess(true)\n",
    "                    .setVerbose(true);\n",
    "        }\n",
    "\n",
    "        System.out.println(Nd4j.getBackend());\n",
    "        Nd4j.getMemoryManager().setAutoGcWindow(5000);\n",
    "\n",
    "        log.info(\"Unfrozen discriminator!\");\n",
    "        ComputationGraph dis = new ComputationGraph(new NeuralNetConfiguration.Builder()\n",
    "                .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .seed(numberOfTheBeast)\n",
    "                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                .gradientNormalizationThreshold(1.0)\n",
    "                .l2(0.0001)\n",
    "                .activation(Activation.TANH)\n",
    "                .weightInit(WeightInit.XAVIER)\n",
    "                .graphBuilder()\n",
    "                .addInputs(\"dis_input_layer_0\")\n",
    "                .setInputTypes(InputType.convolutionalFlat(tensorDimOneSize, tensorDimTwoSize, tensorDimThreeSize))\n",
    "                .addLayer(\"dis_batch_layer_1\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"dis_input_layer_0\")\n",
    "                .addLayer(\"dis_conv2d_layer_2\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(2, 2)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(1)\n",
    "                        .nOut(64)\n",
    "                        .build(), \"dis_batch_layer_1\")\n",
    "                .addLayer(\"dis_maxpool_layer_3\", new SubsamplingLayer.Builder(PoolingType.MAX)\n",
    "                        .kernelSize(2, 2)\n",
    "                        .stride(1, 1)\n",
    "                        .build(), \"dis_conv2d_layer_2\")\n",
    "                .addLayer(\"dis_conv2d_layer_4\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(2, 2)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(64)\n",
    "                        .nOut(128)\n",
    "                        .build(), \"dis_maxpool_layer_3\")\n",
    "                .addLayer(\"dis_maxpool_layer_5\", new SubsamplingLayer.Builder(PoolingType.MAX)\n",
    "                        .kernelSize(2, 2)\n",
    "                        .stride(1, 1)\n",
    "                        .build(), \"dis_conv2d_layer_4\")\n",
    "                .addLayer(\"dis_dense_layer_6\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(1024)\n",
    "                        .build(), \"dis_maxpool_layer_5\")\n",
    "                .addLayer(\"dis_output_layer_7\", new OutputLayer.Builder(LossFunctions.LossFunction.XENT)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(numClassesDis)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .build(), \"dis_dense_layer_6\")\n",
    "                .setOutputs(\"dis_output_layer_7\")\n",
    "                .build());\n",
    "        dis.init();\n",
    "        System.out.println(dis.summary());\n",
    "        System.out.println(Arrays.toString(dis.output(Nd4j.randn(numGenSamples, numFeatures))[0].shape()));\n",
    "\n",
    "        log.info(\"Frozen generator!\");\n",
    "        ComputationGraph gen = new ComputationGraph(new NeuralNetConfiguration.Builder()\n",
    "                .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .seed(numberOfTheBeast)\n",
    "                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                .gradientNormalizationThreshold(1.0)\n",
    "                .l2(0.0001)\n",
    "                .activation(Activation.TANH)\n",
    "                .weightInit(WeightInit.XAVIER)\n",
    "                .graphBuilder()\n",
    "                .addInputs(\"gen_input_layer_0\")\n",
    "                .setInputTypes(InputType.feedForward(zSize))\n",
    "                .addLayer(\"gen_batch_1\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"gen_input_layer_0\")\n",
    "                .addLayer(\"gen_dense_layer_2\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(1024)\n",
    "                        .build(), \"gen_batch_1\")\n",
    "                .addLayer(\"gen_dense_layer_3\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(7 * 7 * 128)\n",
    "                        .build(), \"gen_dense_layer_2\")\n",
    "                .addLayer(\"gen_batch_4\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"gen_dense_layer_3\")\n",
    "                .inputPreProcessor(\"gen_deconv2d_5\", new FeedForwardToCnnPreProcessor(7, 7, 128))\n",
    "                .addLayer(\"gen_deconv2d_5\", new Upsampling2D.Builder(2)\n",
    "                        .build(), \"gen_batch_4\")\n",
    "                .addLayer(\"gen_conv2d_6\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(1, 1)\n",
    "                        .padding(2, 2)\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(128)\n",
    "                        .nOut(64)\n",
    "                        .build(), \"gen_deconv2d_5\")\n",
    "                .addLayer(\"gen_deconv2d_7\", new Upsampling2D.Builder(2)\n",
    "                        .build(), \"gen_conv2d_6\")\n",
    "                .addLayer(\"gen_conv2d_8\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(1, 1)\n",
    "                        .padding(2, 2)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(64)\n",
    "                        .nOut(1)\n",
    "                        .build(), \"gen_deconv2d_7\")\n",
    "                .setOutputs(\"gen_conv2d_8\")\n",
    "                .build());\n",
    "        gen.init();\n",
    "        System.out.println(gen.summary());\n",
    "        System.out.println(Arrays.toString(gen.output(Nd4j.randn(numGenSamples, zSize))[0].shape()));\n",
    "\n",
    "        log.info(\"GAN with unfrozen generator and frozen discriminator!\");\n",
    "        ComputationGraph gan = new ComputationGraph(new NeuralNetConfiguration.Builder()\n",
    "                .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .seed(numberOfTheBeast)\n",
    "                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                .gradientNormalizationThreshold(1.0)\n",
    "                .activation(Activation.TANH)\n",
    "                .weightInit(WeightInit.XAVIER)\n",
    "                .l2(0.0001)\n",
    "                .graphBuilder()\n",
    "                .addInputs(\"gan_input_layer_0\")\n",
    "                .setInputTypes(InputType.feedForward(zSize))\n",
    "                .addLayer(\"gan_batch_1\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"gan_input_layer_0\")\n",
    "                .addLayer(\"gan_dense_layer_2\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(1024)\n",
    "                        .build(), \"gan_batch_1\")\n",
    "                .addLayer(\"gan_dense_layer_3\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(7 * 7 * 128)\n",
    "                        .build(), \"gan_dense_layer_2\")\n",
    "                .addLayer(\"gan_batch_4\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"gan_dense_layer_3\")\n",
    "                .inputPreProcessor(\"gan_deconv2d_5\", new FeedForwardToCnnPreProcessor(7, 7, 128))\n",
    "                .addLayer(\"gan_deconv2d_5\", new Upsampling2D.Builder(2)\n",
    "                        .build(), \"gan_batch_4\")\n",
    "                .addLayer(\"gan_conv2d_6\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(1, 1)\n",
    "                        .padding(2, 2)\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(128)\n",
    "                        .nOut(64)\n",
    "                        .build(), \"gan_deconv2d_5\")\n",
    "                .addLayer(\"gan_deconv2d_7\", new Upsampling2D.Builder(2)\n",
    "                        .build(), \"gan_conv2d_6\")\n",
    "                .addLayer(\"gan_conv2d_8\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(1, 1)\n",
    "                        .padding(2, 2)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(64)\n",
    "                        .nOut(1)\n",
    "                        .build(), \"gan_deconv2d_7\")\n",
    "\n",
    "                .addLayer(\"gan_dis_batch_layer_9\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"gan_conv2d_8\")\n",
    "                .addLayer(\"gan_dis_conv2d_layer_10\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(2, 2)\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(1)\n",
    "                        .nOut(64)\n",
    "                        .build(), \"gan_dis_batch_layer_9\")\n",
    "                .addLayer(\"gan_dis_maxpool_layer_11\", new SubsamplingLayer.Builder(PoolingType.MAX)\n",
    "                        .kernelSize(2, 2)\n",
    "                        .stride(1, 1)\n",
    "                        .build(), \"gan_dis_conv2d_layer_10\")\n",
    "                .addLayer(\"gan_dis_conv2d_layer_12\", new ConvolutionLayer.Builder(5, 5)\n",
    "                        .stride(2, 2)\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(64)\n",
    "                        .nOut(128)\n",
    "                        .build(), \"gan_dis_maxpool_layer_11\")\n",
    "                .addLayer(\"gan_dis_maxpool_layer_13\", new SubsamplingLayer.Builder(PoolingType.MAX)\n",
    "                        .kernelSize(2, 2)\n",
    "                        .stride(1, 1)\n",
    "                        .build(), \"gan_dis_conv2d_layer_12\")\n",
    "                .addLayer(\"gan_dis_dense_layer_14\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(1024)\n",
    "                        .build(), \"gan_dis_maxpool_layer_13\")\n",
    "                .addLayer(\"gan_dis_output_layer_15\", new OutputLayer.Builder(LossFunctions.LossFunction.XENT)\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(numClassesDis)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .build(), \"gan_dis_dense_layer_14\")\n",
    "                .setOutputs(\"gan_dis_output_layer_15\")\n",
    "                .build());\n",
    "        gan.init();\n",
    "        System.out.println(gan.summary());\n",
    "        System.out.println(Arrays.toString(gan.output(Nd4j.randn(numGenSamples, zSize))[0].shape()));\n",
    "\n",
    "        log.info(\"Setting up Spark configuration!\");\n",
    "        SparkConf sparkConf = new SparkConf();\n",
    "        sparkConf.setMaster(\"local[4]\");\n",
    "        sparkConf.setAppName(\"Deeplearning4j on Apache Spark: Generative Adversarial Network!\");\n",
    "        sparkConf.set(\"spark.serializer\", \"org.apache.spark.serializer.KryoSerializer\");\n",
    "        sparkConf.set(\"spark.kryo.registrator\", \"org.nd4j.Nd4jRegistrator\");\n",
    "        JavaSparkContext sc = new JavaSparkContext(sparkConf);\n",
    "\n",
    "        log.info(\"Setting up Synchronous Parameter Averaging!\");\n",
    "        TrainingMaster tm = new ParameterAveragingTrainingMaster.Builder(batchSizePerWorker)\n",
    "                .averagingFrequency(10)\n",
    "                .rngSeed(numberOfTheBeast)\n",
    "                .workerPrefetchNumBatches(0)\n",
    "                .batchSizePerWorker(batchSizePerWorker)\n",
    "                .build();\n",
    "\n",
    "        SparkComputationGraph sparkDis = new SparkComputationGraph(sc, dis, tm);\n",
    "        SparkComputationGraph sparkGan = new SparkComputationGraph(sc, gan, tm);\n",
    "\n",
    "        log.info(\"Computer vision deep learning model with pre-trained layers from the GAN's discriminator!\");\n",
    "        ComputationGraph computerVision = new TransferLearning.GraphBuilder(sparkDis.getNetwork())\n",
    "                .fineTuneConfiguration(new FineTuneConfiguration.Builder()\n",
    "                        .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                        .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                        .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                        .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                        .gradientNormalizationThreshold(1.0)\n",
    "                        .activation(Activation.TANH)\n",
    "                        .l2(0.0001)\n",
    "                        .weightInit(WeightInit.XAVIER)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .seed(numberOfTheBeast)\n",
    "                        .build())\n",
    "                .setFeatureExtractor(\"dis_dense_layer_6\")\n",
    "                .removeVertexKeepConnections(\"dis_output_layer_7\")\n",
    "                .addLayer(\"dis_batch\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(1024)\n",
    "                        .nOut(1024)\n",
    "                        .build(), \"dis_dense_layer_6\")\n",
    "                .addLayer(\"dis_output_layer_7\", new OutputLayer.Builder(LossFunctions.LossFunction.MCXENT)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(1024)\n",
    "                        .nOut(numClasses)\n",
    "                        .activation(Activation.SOFTMAX)\n",
    "                        .build(), \"dis_batch\")\n",
    "                .build();\n",
    "        System.out.println(computerVision.summary());\n",
    "        System.out.println(Arrays.toString(computerVision.output(Nd4j.randn(numGenSamples, numFeatures))[0].shape()));\n",
    "\n",
    "        SparkComputationGraph sparkCV = new SparkComputationGraph(sc, computerVision, tm);\n",
    "\n",
    "        RecordReader recordReaderTrain = new CSVRecordReader(numLinesToSkip, delimiter);\n",
    "        recordReaderTrain.initialize(new FileSplit(new ClassPathResource(dataSetName + \"_train.csv\").getFile()));\n",
    "\n",
    "        DataSetIterator iterTrain = new RecordReaderDataSetIterator(recordReaderTrain, batchSizePerWorker, labelIndex, numClasses);\n",
    "        List<DataSet> trainDataList = new ArrayList<>();\n",
    "\n",
    "        JavaRDD<DataSet> trainDataDis, trainDataGen, trainData;\n",
    "\n",
    "        INDArray grid = Nd4j.linspace(-1.0, 1.0, numGenSamples);\n",
    "        Collection<INDArray> z = new ArrayList<>();\n",
    "        log.info(\"Creating some noise!\");\n",
    "        for (int i = 0; i < numGenSamples; i++) {\n",
    "            for (int j = 0; j < numGenSamples; j++) {\n",
    "                z.add(Nd4j.create(new double[]{grid.getDouble(0, i), grid.getDouble(0, j)}));\n",
    "            }\n",
    "        }\n",
    "\n",
    "        int batch_counter = 0;\n",
    "\n",
    "        DataSet trDataSet;\n",
    "\n",
    "        RecordReader recordReaderTest = new CSVRecordReader(numLinesToSkip, delimiter);\n",
    "        recordReaderTest.initialize(new FileSplit(new ClassPathResource(dataSetName + \"_test.csv\").getFile()));\n",
    "\n",
    "        DataSetIterator iterTest = new RecordReaderDataSetIterator(recordReaderTest, batchSizePred, labelIndex, numClasses);\n",
    "\n",
    "        Collection<INDArray> outFeat;\n",
    "\n",
    "        INDArray out;\n",
    "        INDArray soften_labels_fake = Nd4j.randn(batchSizePerWorker, 1).muli(0.05);\n",
    "        INDArray soften_labels_real = Nd4j.randn(batchSizePerWorker, 1).muli(0.05);\n",
    "\n",
    "        while (iterTrain.hasNext() && batch_counter < numIterations) {\n",
    "            trainDataList.clear();\n",
    "            trDataSet = iterTrain.next();\n",
    "\n",
    "            // This is real data...\n",
    "            // [Fake, Real].\n",
    "            trainDataList.add(new DataSet(trDataSet.getFeatures(), Nd4j.ones(batchSizePerWorker, 1).addi(soften_labels_real)));\n",
    "\n",
    "            // ...and this is fake data.\n",
    "            // [Fake, Real].\n",
    "            trainDataList.add(new DataSet(gen.output(Nd4j.rand(batchSizePerWorker, zSize).muli(2.0).subi(1.0))[0], Nd4j.zeros(batchSizePerWorker, 1).addi(soften_labels_fake)));\n",
    "\n",
    "            // Unfrozen discriminator is trying to figure itself out given a frozen generator.\n",
    "            log.info(\"Training discriminator!\");\n",
    "            trainDataDis = sc.parallelize(trainDataList);\n",
    "            sparkDis.fit(trainDataDis);\n",
    "\n",
    "            // Update GAN's frozen discriminator with unfrozen discriminator.\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_9\").setParam(\"gamma\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"gamma\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_9\").setParam(\"beta\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"beta\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_9\").setParam(\"mean\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"mean\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_9\").setParam(\"var\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"var\"));\n",
    "\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_conv2d_layer_10\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_2\").getParam(\"W\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_conv2d_layer_10\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_2\").getParam(\"b\"));\n",
    "\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_conv2d_layer_12\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_4\").getParam(\"W\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_conv2d_layer_12\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_4\").getParam(\"b\"));\n",
    "\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_dense_layer_14\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_6\").getParam(\"W\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_dense_layer_14\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_6\").getParam(\"b\"));\n",
    "\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_output_layer_15\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_output_layer_7\").getParam(\"W\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_output_layer_15\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_output_layer_7\").getParam(\"b\"));\n",
    "\n",
    "            trainDataList.clear();\n",
    "            // Tell the frozen discriminator that all the fake examples are real examples.\n",
    "            // [Fake, Real].\n",
    "            trainDataList.add(new DataSet(Nd4j.rand(batchSizePerWorker, zSize).muli(2.0).subi(1.0), Nd4j.ones(batchSizePerWorker, 1)));\n",
    "\n",
    "            // Unfrozen generator is trying to fool the frozen discriminator.\n",
    "            log.info(\"Training generator!\");\n",
    "            trainDataGen = sc.parallelize(trainDataList);\n",
    "            sparkGan.fit(trainDataGen);\n",
    "\n",
    "            // Update frozen generator with GAN's unfrozen generator.\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"gamma\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"gamma\"));\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"beta\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"beta\"));\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"mean\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"mean\"));\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"var\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"var\"));\n",
    "\n",
    "            gen.getLayer(\"gen_dense_layer_2\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_2\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_dense_layer_2\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_2\").getParam(\"b\"));\n",
    "\n",
    "            gen.getLayer(\"gen_dense_layer_3\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_3\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_dense_layer_3\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_3\").getParam(\"b\"));\n",
    "\n",
    "            gen.getLayer(\"gen_batch_4\").setParam(\"gamma\", sparkGan.getNetwork().getLayer(\"gan_batch_4\").getParam(\"gamma\"));\n",
    "            gen.getLayer(\"gen_batch_4\").setParam(\"beta\", sparkGan.getNetwork().getLayer(\"gan_batch_4\").getParam(\"beta\"));\n",
    "            gen.getLayer(\"gen_batch_4\").setParam(\"mean\", sparkGan.getNetwork().getLayer(\"gan_batch_4\").getParam(\"mean\"));\n",
    "            gen.getLayer(\"gen_batch_4\").setParam(\"var\", sparkGan.getNetwork().getLayer(\"gan_batch_4\").getParam(\"var\"));\n",
    "\n",
    "            gen.getLayer(\"gen_conv2d_6\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_conv2d_6\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_conv2d_6\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_conv2d_6\").getParam(\"b\"));\n",
    "\n",
    "            gen.getLayer(\"gen_conv2d_8\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_conv2d_8\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_conv2d_8\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_conv2d_8\").getParam(\"b\"));\n",
    "\n",
    "            trainDataList.clear();\n",
    "            trainDataList.add(trDataSet);\n",
    "\n",
    "            log.info(\"Training computer vision model!\");\n",
    "            sparkCV.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"gamma\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"gamma\"));\n",
    "            sparkCV.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"beta\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"beta\"));\n",
    "            sparkCV.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"mean\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"mean\"));\n",
    "            sparkCV.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"var\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"var\"));\n",
    "\n",
    "            sparkCV.getNetwork().getLayer(\"dis_conv2d_layer_2\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_2\").getParam(\"W\"));\n",
    "            sparkCV.getNetwork().getLayer(\"dis_conv2d_layer_2\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_2\").getParam(\"b\"));\n",
    "\n",
    "            sparkCV.getNetwork().getLayer(\"dis_conv2d_layer_4\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_4\").getParam(\"W\"));\n",
    "            sparkCV.getNetwork().getLayer(\"dis_conv2d_layer_4\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_conv2d_layer_4\").getParam(\"b\"));\n",
    "\n",
    "            sparkCV.getNetwork().getLayer(\"dis_dense_layer_6\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_6\").getParam(\"W\"));\n",
    "            sparkCV.getNetwork().getLayer(\"dis_dense_layer_6\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_6\").getParam(\"b\"));\n",
    "\n",
    "            trainData = sc.parallelize(trainDataList);\n",
    "            sparkCV.fit(trainData);\n",
    "\n",
    "            batch_counter++;\n",
    "            log.info(\"Completed Batch {}!\", batch_counter);\n",
    "\n",
    "            if ((batch_counter % printEvery) == 0) {\n",
    "                out = gen.output(Nd4j.vstack(z))[0].reshape(numGenSamples * numGenSamples, numFeatures);\n",
    "\n",
    "                FileWriter fileWriter = new FileWriter(String.format(\"%s%s_out_%d.csv\", resPath, dataSetName, batch_counter));\n",
    "                for (int i = 0; i < out.shape()[0]; i++) {\n",
    "                    for (int j = 0; j < out.shape()[1]; j++) {\n",
    "                        fileWriter.append(String.valueOf(out.getDouble(i, j)));\n",
    "                        if (j != out.shape()[1] - 1) {\n",
    "                            fileWriter.append(delimiter);\n",
    "                        }\n",
    "                    }\n",
    "                    if (i != out.shape()[0] - 1) {\n",
    "                        fileWriter.append(newLine);\n",
    "                    }\n",
    "                }\n",
    "                fileWriter.flush();\n",
    "                fileWriter.close();\n",
    "            }\n",
    "\n",
    "            if ((batch_counter % saveEvery) == 0) {\n",
    "                log.info(\"Ensemble of deep learners for estimation of uncertainty!\");\n",
    "\n",
    "                outFeat = new ArrayList<>();\n",
    "                iterTest.reset();\n",
    "                while (iterTest.hasNext()) {\n",
    "                    outFeat.add(sparkCV.getNetwork().output(iterTest.next().getFeatures())[0]);\n",
    "                }\n",
    "\n",
    "                INDArray toWrite = Nd4j.vstack(outFeat);\n",
    "                FileWriter fileWriter = new FileWriter(String.format(\"%s%s_test_predictions_%d.csv\", resPath, dataSetName, batch_counter));\n",
    "                for (int i = 0; i < toWrite.shape()[0]; i++) {\n",
    "                    for (int j = 0; j < toWrite.shape()[1]; j++) {\n",
    "                        fileWriter.append(String.valueOf(toWrite.getDouble(i, j)));\n",
    "                        if (j != toWrite.shape()[1] - 1) {\n",
    "                            fileWriter.append(delimiter);\n",
    "                        }\n",
    "                    }\n",
    "                    if (i != toWrite.shape()[0] - 1) {\n",
    "                        fileWriter.append(newLine);\n",
    "                    }\n",
    "                }\n",
    "                fileWriter.flush();\n",
    "                fileWriter.close();\n",
    "            }\n",
    "\n",
    "            if (!iterTrain.hasNext()) {\n",
    "                iterTrain.reset();\n",
    "            }\n",
    "        }\n",
    "\n",
    "        log.info(\"Saving models!\");\n",
    "        ModelSerializer.writeModel(sparkDis.getNetwork(), new File(resPath + dataSetName + \"_dis_model.zip\"), true);\n",
    "        ModelSerializer.writeModel(sparkGan.getNetwork(), new File(resPath + dataSetName + \"_gan_model.zip\"), true);\n",
    "        ModelSerializer.writeModel(gen, new File(resPath + dataSetName + \"_gen_model.zip\"), true);\n",
    "        ModelSerializer.writeModel(sparkCV.getNetwork(), new File(resPath + dataSetName + \"_CV_model.zip\"), true);\n",
    "\n",
    "        tm.deleteTempFiles(sc);\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_path = \"/Users/samson/Projects/gan_deeplearning4j/outputs/computer_vision/\"\n",
    "out_feat = pd.read_csv(filepath_or_buffer=output_path + \"mnist_test_predictions_10000.csv\",\n",
    "                       header=None)\n",
    "acc_dcgan = sum(out_feat.idxmax(axis=1) == y_test) / y_test.shape[0]\n",
    "print(\"The accuracy score for the MNIST classification task with DCGAN: %.6f%%.\" % (acc_dcgan * 100))\n",
    "\n",
    "out = pd.read_csv(filepath_or_buffer=output_path + \"out_10000.csv\", header=None)\n",
    "digit_size = 28\n",
    "n = 10\n",
    "figure = np.zeros((digit_size * n, digit_size * n))\n",
    "\n",
    "counter = 0\n",
    "for i in range(n):\n",
    "    for j in range(n):\n",
    "        figure[i * digit_size: (i + 1) * digit_size, j * digit_size: (j + 1) * digit_size] = out.iloc[counter].reshape(digit_size, digit_size)\n",
    "        counter = counter + 1\n",
    "        \n",
    "plt.figure(figsize=(20, 20))\n",
    "plt.title(\"Deep Convolutional Generative Adversarial Network (DCGAN) with a 2-dimensional latent manifold\\nGenerating new images on the 2-dimensional latent manifold\", fontsize=20)\n",
    "plt.xlabel(\"Latent dimension 1\", fontsize=24)\n",
    "plt.ylabel(\"Latent dimension 2\", fontsize=24)\n",
    "plt.imshow(figure, cmap=\"Greys_r\")\n",
    "plt.savefig(fname=\"DCGAN_Generated_Images.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "claim_risk = pd.read_csv(filepath_or_buffer=\"data/claim_risk.csv\")\n",
    "claim_risk.drop(columns=\"policy.id\", axis=1, inplace=True)\n",
    "claim_risk = np.asarray(claim_risk).ravel()\n",
    "\n",
    "transactions = pd.read_csv(filepath_or_buffer=\"data/transactions.csv\")\n",
    "transactions.drop(columns=\"policy.id\", axis=1, inplace=True)\n",
    "\n",
    "n_policies = 1000\n",
    "n_transaction_types = 3\n",
    "n_time_periods = 4\n",
    "\n",
    "transactions = np.reshape(np.asarray(transactions), (n_policies, n_time_periods * n_transaction_types))\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(transactions, claim_risk, test_size=0.3, random_state=666)\n",
    "\n",
    "min_X_train = np.apply_along_axis(func1d=np.min, axis=0, arr=X_train)\n",
    "max_X_train = np.apply_along_axis(func1d=np.max, axis=0, arr=X_train) \n",
    "range_X_train = max_X_train - min_X_train + sys.float_info.epsilon\n",
    "X_train = (X_train - min_X_train) / range_X_train\n",
    "X_test = (X_test - min_X_train) / range_X_train\n",
    "transactions = (transactions - min_X_train) / range_X_train\n",
    "\n",
    "X_train = np.reshape(np.asarray(X_train), (X_train.shape[0], n_time_periods, n_transaction_types, 1))\n",
    "X_test = np.reshape(np.asarray(X_test), (X_test.shape[0], n_time_periods, n_transaction_types, 1))\n",
    "transactions = np.reshape(np.asarray(transactions), (n_policies, n_time_periods, n_transaction_types, 1))\n",
    "\n",
    "np.savetxt(fname=dl4j_res_path + \"insurance_train.csv\",\n",
    "           X=np.hstack([X_train.reshape([X_train.shape[0], n_time_periods * n_transaction_types]), \n",
    "                        y_train.reshape([-1, 1])]),\n",
    "           fmt=\"%.2f\",\n",
    "           delimiter=\",\",\n",
    "           newline=\"\\n\",\n",
    "           header=\"\")\n",
    "\n",
    "np.savetxt(fname=dl4j_res_path + \"insurance_test.csv\",\n",
    "           X=np.hstack([X_test.reshape([X_test.shape[0], n_time_periods * n_transaction_types]),\n",
    "                        y_test.reshape([-1, 1])]),\n",
    "           fmt=\"%.2f\",\n",
    "           delimiter=\",\",\n",
    "           newline=\"\\n\",\n",
    "           header=\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Java code for insurance task\n",
    "\n",
    "```java\n",
    "package org.deeplearning4j;\n",
    "\n",
    "import java.io.*;\n",
    "import java.util.*;\n",
    "\n",
    "import org.apache.spark.SparkConf;\n",
    "import org.apache.spark.api.java.JavaRDD;\n",
    "import org.apache.spark.api.java.JavaSparkContext;\n",
    "\n",
    "import org.datavec.api.records.reader.RecordReader;\n",
    "import org.datavec.api.records.reader.impl.csv.CSVRecordReader;\n",
    "import org.datavec.api.split.FileSplit;\n",
    "import org.datavec.api.util.ClassPathResource;\n",
    "\n",
    "import org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator;\n",
    "import org.deeplearning4j.nn.api.*;\n",
    "import org.deeplearning4j.nn.conf.GradientNormalization;\n",
    "import org.deeplearning4j.nn.conf.WorkspaceMode;\n",
    "import org.deeplearning4j.nn.conf.inputs.InputType;\n",
    "import org.deeplearning4j.nn.conf.layers.*;\n",
    "import org.deeplearning4j.nn.conf.NeuralNetConfiguration;\n",
    "import org.deeplearning4j.nn.graph.ComputationGraph;\n",
    "import org.deeplearning4j.nn.transferlearning.*;\n",
    "import org.deeplearning4j.nn.weights.*;\n",
    "import org.deeplearning4j.spark.api.TrainingMaster;\n",
    "import org.deeplearning4j.spark.impl.graph.SparkComputationGraph;\n",
    "import org.deeplearning4j.spark.impl.paramavg.ParameterAveragingTrainingMaster;\n",
    "import org.deeplearning4j.util.ModelSerializer;\n",
    "\n",
    "import org.nd4j.jita.conf.CudaEnvironment;\n",
    "import org.nd4j.linalg.activations.*;\n",
    "import org.nd4j.linalg.api.buffer.DataBuffer;\n",
    "import org.nd4j.linalg.api.ndarray.INDArray;\n",
    "import org.nd4j.linalg.dataset.DataSet;\n",
    "import org.nd4j.linalg.dataset.api.iterator.DataSetIterator;\n",
    "import org.nd4j.linalg.factory.Nd4j;\n",
    "import org.nd4j.linalg.learning.config.*;\n",
    "import org.nd4j.linalg.lossfunctions.*;\n",
    "\n",
    "import org.slf4j.Logger;\n",
    "import org.slf4j.LoggerFactory;\n",
    "\n",
    "public class dl4jGANInsurance {\n",
    "    private static final Logger log = LoggerFactory.getLogger(dl4jGANInsurance.class);\n",
    "\n",
    "    private static final int batchSizePerWorker = 50;\n",
    "    private static final int batchSizePred = 700;\n",
    "    private static final int labelIndex = 12;\n",
    "    private static final int numClasses = 1; // Using Sigmoid.\n",
    "    private static final int numClassesDis = 1; // Using Sigmoid.\n",
    "    private static final int numFeatures = 12;\n",
    "    private static final int numIterations = 5000;\n",
    "    private static final int numGenSamples = 50; // This will be a grid so effectively we get {numGenSamples * numGenSamples} samples.\n",
    "    private static final int numLinesToSkip = 0;\n",
    "    private static final int numberOfTheBeast = 666;\n",
    "    private static final int printEvery = 100;\n",
    "    private static final int saveEvery = 100;\n",
    "    private static final int tensorDimOneSize = 4;\n",
    "    private static final int tensorDimTwoSize = 3;\n",
    "    private static final int tensorDimThreeSize = 1;\n",
    "    private static final int zSize = 2;\n",
    "\n",
    "    private static final double dis_learning_rate = 0.0002;\n",
    "    private static final double frozen_learning_rate = 0.0;\n",
    "    private static final double gen_learning_rate = 0.0004;\n",
    "\n",
    "    private static final String delimiter = \",\";\n",
    "    private static final String resPath = \"/Users/samson/Projects/gan_deeplearning4j/outputs/insurance/\";\n",
    "    private static final String newLine = \"\\n\";\n",
    "    private static final String dataSetName = \"insurance\";\n",
    "\n",
    "    private static final boolean useGpu = false;\n",
    "\n",
    "    public static void main(String[] args) throws Exception {\n",
    "        new dl4jGANInsurance().GAN(args);\n",
    "    }\n",
    "\n",
    "    private void GAN(String[] args) throws Exception {\n",
    "        for (int i = 0; i < args.length; i++) {\n",
    "            System.out.println(args[i]);\n",
    "        }\n",
    "\n",
    "        if (useGpu) {\n",
    "            System.out.println(\"Setting up CUDA environment!\");\n",
    "            Nd4j.setDataType(DataBuffer.Type.FLOAT);\n",
    "\n",
    "            CudaEnvironment.getInstance().getConfiguration()\n",
    "                    .allowMultiGPU(true)\n",
    "                    .setMaximumDeviceCache(2L * 1024L * 1024L * 1024L)\n",
    "                    .allowCrossDeviceAccess(true)\n",
    "                    .setVerbose(true);\n",
    "        }\n",
    "\n",
    "        System.out.println(Nd4j.getBackend());\n",
    "        Nd4j.getMemoryManager().setAutoGcWindow(5000);\n",
    "\n",
    "        log.info(\"Unfrozen discriminator!\");\n",
    "        ComputationGraph dis = new ComputationGraph(new NeuralNetConfiguration.Builder()\n",
    "                .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .seed(numberOfTheBeast)\n",
    "                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                .gradientNormalizationThreshold(1.0)\n",
    "                .l2(0.0001)\n",
    "                .activation(Activation.ELU)\n",
    "                .weightInit(WeightInit.XAVIER)\n",
    "                .graphBuilder()\n",
    "                .addInputs(\"dis_input_layer_0\")\n",
    "                .addLayer(\"dis_batch_layer_1\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(tensorDimOneSize * tensorDimTwoSize * tensorDimThreeSize)\n",
    "                        .nOut(tensorDimOneSize * tensorDimTwoSize * tensorDimThreeSize)\n",
    "                        .build(), \"dis_input_layer_0\")\n",
    "                .addLayer(\"dis_dense_layer_2\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(tensorDimOneSize * tensorDimTwoSize * tensorDimThreeSize)\n",
    "                        .nOut(100)\n",
    "                        .build(),\"dis_batch_layer_1\")\n",
    "                .addLayer(\"dis_dropout_layer_3\", new DropoutLayer(),\n",
    "                        \"dis_dense_layer_2\")\n",
    "                .addLayer(\"dis_output_layer_4\", new OutputLayer.Builder(LossFunctions.LossFunction.XENT)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(100)\n",
    "                        .nOut(numClassesDis)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .build(), \"dis_dropout_layer_3\")\n",
    "                .setOutputs(\"dis_output_layer_4\")\n",
    "                .build());\n",
    "        dis.init();\n",
    "        System.out.println(dis.summary());\n",
    "        System.out.println(Arrays.toString(dis.output(Nd4j.randn(numGenSamples, numFeatures))[0].shape()));\n",
    "\n",
    "        log.info(\"Frozen generator!\");\n",
    "        ComputationGraph gen = new ComputationGraph(new NeuralNetConfiguration.Builder()\n",
    "                .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .seed(numberOfTheBeast)\n",
    "                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                .gradientNormalizationThreshold(1.0)\n",
    "                .l2(0.0001)\n",
    "                .activation(Activation.TANH)\n",
    "                .weightInit(WeightInit.XAVIER)\n",
    "                .graphBuilder()\n",
    "                .addInputs(\"gen_input_layer_0\")\n",
    "                .setInputTypes(InputType.feedForward(zSize))\n",
    "                .addLayer(\"gen_batch_1\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"gen_input_layer_0\")\n",
    "                .addLayer(\"gen_dense_layer_2\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(100)\n",
    "                        .build(), \"gen_batch_1\")\n",
    "                .addLayer(\"gen_dense_layer_3\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(100)\n",
    "                        .build(), \"gen_dense_layer_2\")\n",
    "                .addLayer(\"gen_dense_layer_4\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(100)\n",
    "                        .build(), \"gen_dense_layer_3\")\n",
    "                .addLayer(\"gen_dense_layer_5\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(100)\n",
    "                        .nOut(tensorDimOneSize * tensorDimTwoSize * tensorDimThreeSize)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .build(), \"gen_dense_layer_4\")\n",
    "                .setOutputs(\"gen_dense_layer_5\")\n",
    "                .build());\n",
    "        gen.init();\n",
    "        System.out.println(gen.summary());\n",
    "        System.out.println(Arrays.toString(gen.output(Nd4j.randn(numGenSamples, zSize))[0].reshape(numGenSamples, tensorDimOneSize, tensorDimTwoSize, tensorDimThreeSize).shape()));\n",
    "\n",
    "        log.info(\"GAN with unfrozen generator and frozen discriminator!\");\n",
    "        ComputationGraph gan = new ComputationGraph(new NeuralNetConfiguration.Builder()\n",
    "                .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                .seed(numberOfTheBeast)\n",
    "                .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                .gradientNormalizationThreshold(1.0)\n",
    "                .activation(Activation.TANH)\n",
    "                .weightInit(WeightInit.XAVIER)\n",
    "                .l2(0.0001)\n",
    "                .graphBuilder()\n",
    "                .addInputs(\"gan_input_layer_0\")\n",
    "                .setInputTypes(InputType.feedForward(zSize))\n",
    "                .addLayer(\"gan_batch_1\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .build(), \"gan_input_layer_0\")\n",
    "                .addLayer(\"gan_dense_layer_2\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(100)\n",
    "                        .build(), \"gan_batch_1\")\n",
    "                .addLayer(\"gan_dense_layer_3\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(100)\n",
    "                        .build(), \"gan_dense_layer_2\")\n",
    "                .addLayer(\"gan_dense_layer_4\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(100)\n",
    "                        .build(), \"gan_dense_layer_3\")\n",
    "                .addLayer(\"gan_dense_layer_5\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(gen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(tensorDimOneSize * tensorDimTwoSize * tensorDimThreeSize)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .build(), \"gan_dense_layer_4\")\n",
    "\n",
    "                .addLayer(\"gan_dis_batch_layer_6\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .activation(Activation.ELU)\n",
    "                        .build(), \"gan_dense_layer_5\")\n",
    "                .addLayer(\"gan_dis_dense_layer_7\", new DenseLayer.Builder()\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .activation(Activation.ELU)\n",
    "                        .nIn(tensorDimOneSize * tensorDimTwoSize * tensorDimThreeSize)\n",
    "                        .nOut(100)\n",
    "                        .build(),\"gan_dis_batch_layer_6\")\n",
    "                .addLayer(\"gan_dis_dropout_layer_8\", new DropoutLayer(),\n",
    "                        \"gan_dis_dense_layer_7\")\n",
    "                .addLayer(\"gan_dis_output_layer_9\", new OutputLayer.Builder(LossFunctions.LossFunction.XENT)\n",
    "                        .updater(new RmsProp(frozen_learning_rate, 1e-8, 1e-8))\n",
    "                        .nOut(numClassesDis)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .build(), \"gan_dis_dropout_layer_8\")\n",
    "                .setOutputs(\"gan_dis_output_layer_9\")\n",
    "                .build());\n",
    "        gan.init();\n",
    "        System.out.println(gan.summary());\n",
    "        System.out.println(Arrays.toString(gan.output(Nd4j.randn(numGenSamples, zSize))[0].shape()));\n",
    "\n",
    "        log.info(\"Setting up Spark configuration!\");\n",
    "        SparkConf sparkConf = new SparkConf();\n",
    "        sparkConf.setMaster(\"local[4]\");\n",
    "        sparkConf.setAppName(\"Deeplearning4j on Apache Spark: Generative Adversarial Network!\");\n",
    "        sparkConf.set(\"spark.serializer\", \"org.apache.spark.serializer.KryoSerializer\");\n",
    "        sparkConf.set(\"spark.kryo.registrator\", \"org.nd4j.Nd4jRegistrator\");\n",
    "        JavaSparkContext sc = new JavaSparkContext(sparkConf);\n",
    "\n",
    "        log.info(\"Setting up Synchronous Parameter Averaging!\");\n",
    "        TrainingMaster tm = new ParameterAveragingTrainingMaster.Builder(batchSizePerWorker)\n",
    "                .averagingFrequency(5)\n",
    "                .rngSeed(numberOfTheBeast)\n",
    "                .workerPrefetchNumBatches(0)\n",
    "                .batchSizePerWorker(batchSizePerWorker)\n",
    "                .build();\n",
    "\n",
    "        SparkComputationGraph sparkDis = new SparkComputationGraph(sc, dis, tm);\n",
    "        SparkComputationGraph sparkGan = new SparkComputationGraph(sc, gan, tm);\n",
    "\n",
    "        log.info(\"Insurance deep learning model with pre-trained layers from the GAN's discriminator!\");\n",
    "        ComputationGraph insurance = new TransferLearning.GraphBuilder(sparkDis.getNetwork())\n",
    "                .fineTuneConfiguration(new FineTuneConfiguration.Builder()\n",
    "                        .trainingWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                        .inferenceWorkspaceMode(WorkspaceMode.ENABLED)\n",
    "                        .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)\n",
    "                        .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)\n",
    "                        .gradientNormalizationThreshold(1.0)\n",
    "                        .activation(Activation.ELU)\n",
    "                        .l2(0.0001)\n",
    "                        .weightInit(WeightInit.XAVIER)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .seed(numberOfTheBeast)\n",
    "                        .build())\n",
    "                .setFeatureExtractor(\"dis_dropout_layer_3\")\n",
    "                .removeVertexKeepConnections(\"dis_output_layer_4\")\n",
    "                .addLayer(\"dis_batch\", new BatchNormalization.Builder()\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(100)\n",
    "                        .nOut(100)\n",
    "                        .build(), \"dis_dropout_layer_3\")\n",
    "                .addLayer(\"dis_output_layer_4\", new OutputLayer.Builder(LossFunctions.LossFunction.XENT)\n",
    "                        .updater(new RmsProp(dis_learning_rate, 1e-8, 1e-8))\n",
    "                        .nIn(100)\n",
    "                        .nOut(numClasses)\n",
    "                        .activation(Activation.SIGMOID)\n",
    "                        .build(), \"dis_batch\")\n",
    "                .build();\n",
    "        System.out.println(insurance.summary());\n",
    "        System.out.println(Arrays.toString(insurance.output(Nd4j.randn(numGenSamples, numFeatures))[0].shape()));\n",
    "\n",
    "        SparkComputationGraph sparkInsurance = new SparkComputationGraph(sc, insurance, tm);\n",
    "\n",
    "        RecordReader recordReaderTrain = new CSVRecordReader(numLinesToSkip, delimiter);\n",
    "        recordReaderTrain.initialize(new FileSplit(new ClassPathResource(dataSetName + \"_train.csv\").getFile()));\n",
    "\n",
    "        DataSetIterator iterTrain = new RecordReaderDataSetIterator(recordReaderTrain, batchSizePerWorker, labelIndex, numClasses);\n",
    "        List<DataSet> trainDataList = new ArrayList<>();\n",
    "\n",
    "        JavaRDD<DataSet> trainDataDis, trainDataGen, trainData;\n",
    "\n",
    "        INDArray grid = Nd4j.linspace(-1.0, 1.0, numGenSamples);\n",
    "        Collection<INDArray> z = new ArrayList<>();\n",
    "        log.info(\"Creating some noise!\");\n",
    "        for (int i = 0; i < numGenSamples; i++) {\n",
    "            for (int j = 0; j < numGenSamples; j++) {\n",
    "                z.add(Nd4j.create(new double[]{grid.getDouble(0, i), grid.getDouble(0, j)}));\n",
    "            }\n",
    "        }\n",
    "\n",
    "        int batch_counter = 0;\n",
    "\n",
    "        DataSet trDataSet;\n",
    "\n",
    "        RecordReader recordReaderTest = new CSVRecordReader(numLinesToSkip, delimiter);\n",
    "        recordReaderTest.initialize(new FileSplit(new ClassPathResource(dataSetName + \"_test.csv\").getFile()));\n",
    "\n",
    "        DataSetIterator iterTest = new RecordReaderDataSetIterator(recordReaderTest, batchSizePred, labelIndex, numClasses);\n",
    "\n",
    "        Collection<INDArray> outFeat;\n",
    "\n",
    "        INDArray out, outPred;\n",
    "        INDArray soften_labels_fake = Nd4j.randn(batchSizePerWorker, 1).muli(0.05);\n",
    "        INDArray soften_labels_real = Nd4j.randn(batchSizePerWorker, 1).muli(0.05);\n",
    "        \n",
    "        while (iterTrain.hasNext() && batch_counter < numIterations) {\n",
    "            trainDataList.clear();\n",
    "            trDataSet = iterTrain.next();\n",
    "\n",
    "            // This is real data...\n",
    "            // [Fake, Real].\n",
    "            trainDataList.add(new DataSet(trDataSet.getFeatures(), Nd4j.ones(batchSizePerWorker, 1).addi(soften_labels_real)));\n",
    "\n",
    "            // ...and this is fake data.\n",
    "            // [Fake, Real].\n",
    "            trainDataList.add(new DataSet(gen.output(Nd4j.rand(batchSizePerWorker, zSize).muli(2.0).subi(1.0))[0], Nd4j.zeros(batchSizePerWorker, 1).addi(soften_labels_fake)));\n",
    "\n",
    "            // Unfrozen discriminator is trying to figure itself out given a frozen generator.\n",
    "            log.info(\"Training discriminator!\");\n",
    "            trainDataDis = sc.parallelize(trainDataList);\n",
    "            sparkDis.fit(trainDataDis);\n",
    "\n",
    "            // Update GAN's frozen discriminator with unfrozen discriminator.\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_6\").setParam(\"gamma\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"gamma\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_6\").setParam(\"beta\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"beta\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_6\").setParam(\"mean\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"mean\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_batch_layer_6\").setParam(\"var\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"var\"));\n",
    "\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_dense_layer_7\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_2\").getParam(\"W\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_dense_layer_7\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_2\").getParam(\"b\"));\n",
    "\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_output_layer_9\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_output_layer_4\").getParam(\"W\"));\n",
    "            sparkGan.getNetwork().getLayer(\"gan_dis_output_layer_9\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_output_layer_4\").getParam(\"b\"));\n",
    "\n",
    "            trainDataList.clear();\n",
    "            // Tell the frozen discriminator that all the fake examples are real examples.\n",
    "            // [Fake, Real].\n",
    "            trainDataList.add(new DataSet(Nd4j.rand(batchSizePerWorker, zSize).muli(2.0).subi(1.0), Nd4j.ones(batchSizePerWorker, 1)));\n",
    "\n",
    "            // Unfrozen generator is trying to fool the frozen discriminator.\n",
    "            log.info(\"Training generator!\");\n",
    "            trainDataGen = sc.parallelize(trainDataList);\n",
    "            sparkGan.fit(trainDataGen);\n",
    "\n",
    "            // Update frozen generator with GAN's unfrozen generator.\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"gamma\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"gamma\"));\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"beta\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"beta\"));\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"mean\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"mean\"));\n",
    "            gen.getLayer(\"gen_batch_1\").setParam(\"var\", sparkGan.getNetwork().getLayer(\"gan_batch_1\").getParam(\"var\"));\n",
    "\n",
    "            gen.getLayer(\"gen_dense_layer_2\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_2\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_dense_layer_2\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_2\").getParam(\"b\"));\n",
    "\n",
    "            gen.getLayer(\"gen_dense_layer_3\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_3\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_dense_layer_3\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_3\").getParam(\"b\"));\n",
    "\n",
    "            gen.getLayer(\"gen_dense_layer_4\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_4\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_dense_layer_4\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_4\").getParam(\"b\"));\n",
    "\n",
    "            gen.getLayer(\"gen_dense_layer_5\").setParam(\"W\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_5\").getParam(\"W\"));\n",
    "            gen.getLayer(\"gen_dense_layer_5\").setParam(\"b\", sparkGan.getNetwork().getLayer(\"gan_dense_layer_5\").getParam(\"b\"));\n",
    "\n",
    "            trainDataList.clear();\n",
    "            trainDataList.add(trDataSet);\n",
    "\n",
    "            log.info(\"Training insurance model!\");\n",
    "            sparkInsurance.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"gamma\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"gamma\"));\n",
    "            sparkInsurance.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"beta\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"beta\"));\n",
    "            sparkInsurance.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"mean\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"mean\"));\n",
    "            sparkInsurance.getNetwork().getLayer(\"dis_batch_layer_1\").setParam(\"var\", sparkDis.getNetwork().getLayer(\"dis_batch_layer_1\").getParam(\"var\"));\n",
    "\n",
    "            sparkInsurance.getNetwork().getLayer(\"dis_dense_layer_2\").setParam(\"W\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_2\").getParam(\"W\"));\n",
    "            sparkInsurance.getNetwork().getLayer(\"dis_dense_layer_2\").setParam(\"b\", sparkDis.getNetwork().getLayer(\"dis_dense_layer_2\").getParam(\"b\"));\n",
    "\n",
    "            trainData = sc.parallelize(trainDataList);\n",
    "            sparkInsurance.fit(trainData);\n",
    "\n",
    "            batch_counter++;\n",
    "            log.info(\"Completed Batch {}!\", batch_counter);\n",
    "\n",
    "            if ((batch_counter % printEvery) == 0) {\n",
    "                out = gen.output(Nd4j.vstack(z))[0].reshape(numGenSamples * numGenSamples, numFeatures);\n",
    "\n",
    "                FileWriter fileWriter = new FileWriter(String.format(\"%s%s_out_%d.csv\", resPath, dataSetName, batch_counter));\n",
    "                for (int i = 0; i < out.shape()[0]; i++) {\n",
    "                    for (int j = 0; j < out.shape()[1]; j++) {\n",
    "                        fileWriter.append(String.valueOf(out.getDouble(i, j)));\n",
    "                        if (j != out.shape()[1] - 1) {\n",
    "                            fileWriter.append(delimiter);\n",
    "                        }\n",
    "                    }\n",
    "                    if (i != out.shape()[0] - 1) {\n",
    "                        fileWriter.append(newLine);\n",
    "                    }\n",
    "                }\n",
    "                fileWriter.flush();\n",
    "                fileWriter.close();\n",
    "\n",
    "                outPred = sparkInsurance.getNetwork().output(out)[0];\n",
    "\n",
    "                fileWriter = new FileWriter(String.format(\"%s%s_out_pred_%d.csv\", resPath, dataSetName, batch_counter));\n",
    "                for (int i = 0; i < outPred.shape()[0]; i++) {\n",
    "                    for (int j = 0; j < outPred.shape()[1]; j++) {\n",
    "                        fileWriter.append(String.valueOf(outPred.getDouble(i, j)));\n",
    "                        if (j != outPred.shape()[1] - 1) {\n",
    "                            fileWriter.append(delimiter);\n",
    "                        }\n",
    "                    }\n",
    "                    if (i != outPred.shape()[0] - 1) {\n",
    "                        fileWriter.append(newLine);\n",
    "                    }\n",
    "                }\n",
    "                fileWriter.flush();\n",
    "                fileWriter.close();\n",
    "            }\n",
    "\n",
    "            if ((batch_counter % saveEvery) == 0) {\n",
    "                log.info(\"Ensemble of deep learners for estimation of uncertainty!\");\n",
    "\n",
    "                outFeat = new ArrayList<>();\n",
    "                iterTest.reset();\n",
    "                while (iterTest.hasNext()) {\n",
    "                    outFeat.add(sparkInsurance.getNetwork().output(iterTest.next().getFeatures())[0]);\n",
    "                }\n",
    "\n",
    "                INDArray toWrite = Nd4j.vstack(outFeat);\n",
    "                FileWriter fileWriter = new FileWriter(String.format(\"%s%s_test_predictions_%d.csv\", resPath, dataSetName, batch_counter));\n",
    "                for (int i = 0; i < toWrite.shape()[0]; i++) {\n",
    "                    for (int j = 0; j < toWrite.shape()[1]; j++) {\n",
    "                        fileWriter.append(String.valueOf(toWrite.getDouble(i, j)));\n",
    "                        if (j != toWrite.shape()[1] - 1) {\n",
    "                            fileWriter.append(delimiter);\n",
    "                        }\n",
    "                    }\n",
    "                    if (i != toWrite.shape()[0] - 1) {\n",
    "                        fileWriter.append(newLine);\n",
    "                    }\n",
    "                }\n",
    "                fileWriter.flush();\n",
    "                fileWriter.close();\n",
    "            }\n",
    "\n",
    "            if (!iterTrain.hasNext()) {\n",
    "                iterTrain.reset();\n",
    "            }\n",
    "        }\n",
    "\n",
    "        log.info(\"Saving models!\");\n",
    "        ModelSerializer.writeModel(sparkDis.getNetwork(), new File(resPath + dataSetName + \"_dis_model.zip\"), true);\n",
    "        ModelSerializer.writeModel(sparkGan.getNetwork(), new File(resPath + dataSetName + \"_gan_model.zip\"), true);\n",
    "        ModelSerializer.writeModel(gen, new File(resPath + dataSetName + \"_gen_model.zip\"), true);\n",
    "        ModelSerializer.writeModel(sparkInsurance.getNetwork(), new File(resPath + dataSetName + \"_insurance_model.zip\"), true);\n",
    "\n",
    "        tm.deleteTempFiles(sc);\n",
    "    }\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_path = \"/Users/samson/Projects/gan_deeplearning4j/outputs/insurance/\"\n",
    "out_feat = pd.read_csv(filepath_or_buffer=output_path + \"insurance_test_predictions_1000.csv\",\n",
    "                       header=None)\n",
    "auroc_dcgan_ins = roc_auc_score(y_true=y_test,\n",
    "                                y_score=out_feat, \n",
    "                                average=\"weighted\")\n",
    "print(\"The AUROC score for the insurance classification task with DCGAN: %.6f%%.\" % (auroc_dcgan_ins * 100))\n",
    "\n",
    "out = pd.read_csv(filepath_or_buffer=output_path + \"insurance_out_1000.csv\", header=None)\n",
    "out_pred = pd.read_csv(filepath_or_buffer=output_path + \"insurance_out_pred_1000.csv\", header=None)\n",
    "\n",
    "n = 50\n",
    "figure = np.zeros((n_time_periods * n, n_transaction_types * n))\n",
    "plot_preds = np.zeros((n, n))\n",
    "counter = 0\n",
    "for i in range(n):\n",
    "    for j in range(n):\n",
    "        figure[i * n_time_periods: (i + 1) * n_time_periods, j * n_transaction_types: (j + 1) * n_transaction_types] = out.iloc[counter].reshape(n_time_periods, n_transaction_types)\n",
    "        plot_preds[i: (i + 1), j: (j + 1)] = out_pred.iloc[counter]\n",
    "        counter = counter + 1\n",
    "        \n",
    "fig = plt.figure(figsize=(20, 30))\n",
    "fig.add_subplot(2, 1, 1)\n",
    "plt.imshow(plot_preds, cmap=\"hot_r\")\n",
    "plt.title(\"Deep Convolutional Generative Adversarial Network (DCGAN) with a 2-dimensional latent manifold for the insurance data\\nPredicted probability of insurance loss: Darker means higher probability\", fontsize=20)\n",
    "plt.xlabel(\"Latent dimension 1\", fontsize=24)\n",
    "plt.ylabel(\"Latent dimension 2\", fontsize=24)\n",
    "fig.add_subplot(2, 1, 2)\n",
    "plt.imshow(figure, cmap=\"Greys\")\n",
    "plt.title(\"Deep Convolutional Generative Adversarial Network (DCGAN) with a 2-dimensional latent manifold for the insurance data\\nGenerating new transactions data on the 2-dimensional latent manifold\", fontsize=20)\n",
    "plt.xlabel(\"Latent dimension 1\", fontsize=24)\n",
    "plt.ylabel(\"Latent dimension 2\", fontsize=24)\n",
    "plt.savefig(fname=\"DCGAN_Generated_Lattices.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "---\n",
    "\n",
    "We have shown how to use GANs to learn a good representation of raw data, i.e., 1 or 2 dimensional tensors per unit of analysis, that can then perhaps be used for supervised learning tasks in the domain of computer vision and insurance. This moves us away from manual handcrafted feature engineering towards automatic feature engineering, i.e., representation learning. GANs can perhaps be also used for semi-supervised learning which will be the topic of another paper."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References\n",
    "\n",
    "---\n",
    "\n",
    "1. Goodfellow, I., Bengio, Y. and Courville A. (2016). Deep Learning (MIT Press).\n",
    "2. Geron, A. (2017). Hands-On Machine Learning with Scikit-Learn & Tensorflow (O'Reilly).\n",
    "3. Radford, A., Luke, M. and Chintala, S. (2015). Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (https://arxiv.org/abs/1511.06434).\n",
    "4. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y. (2014). Generative Adversarial Networks (https://arxiv.org/abs/1406.2661).\n",
    "5. http://scikit-learn.org/stable/#\n",
    "6. https://towardsdatascience.com/learning-rate-schedules-and-adaptive-learning-rate-methods-for-deep-learning-2c8f433990d1\n",
    "7. https://stackoverflow.com/questions/42177658/how-to-switch-backend-with-keras-from-tensorflow-to-theano\n",
    "8. https://blog.keras.io/building-autoencoders-in-keras.html\n",
    "9. https://keras.io\n",
    "10. https://github.com/fchollet/keras/blob/master/examples/mnist_acgan.py#L24\n",
    "11. https://en.wikipedia.org/wiki/Kullback–Leibler_divergence\n",
    "12. https://see.stanford.edu/materials/lsocoee364b/01-subgradients_notes.pdf\n",
    "13. https://blog.skymind.ai/distributed-deep-learning-part-1-an-introduction-to-distributed-training-of-neural-networks/\n",
    "14. https://deeplearning4j.org\n",
    "15. https://github.com/hamaadshah/gan_keras\n",
    "16. https://towardsdatascience.com/automatic-feature-engineering-using-generative-adversarial-networks-8e24b3c16bf3"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
